In [37]:
!mkdir -p ~/agave
%cd ~/agave
!pip2 install --upgrade setvar
import re
import os
import sys
from setvar import *
from time import sleep
# This cell enables inline plotting in the notebook
%matplotlib inline
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
loadvar()
In this notebook, we introduce some basic uses of the agavepy Python library for interacting with the Agave Platform science-as-a-service APIs. The examples primarily draw from the apps service, but the concepts introduced are broadly applicable to all Agave services. In subsequent notebooks, we'll take deeper dives into specific topics such as using agavepy to launch and monitor an Agave job. For more information about Agave, please see the developer site: http://agaveapi.co/
The agavepy library provides a high-level Python binding to the Agave API. The first step is to import the Agave class:
In [33]:
from agavepy.agave import Agave
import json
Before we can interact with Agave, we need to instantiate a client. Typically, we would use the constructor and pass in our credentials (OAuth client key and secret as well as our username and password) together with configuration data for our "tenant", the organization within Agave we wish to interact with.
In [34]:
agave_cache_dir = os.environ.get('AGAVE_CACHE_DIR')
ag_token_cache = json.loads(open(agave_cache_dir + '/current').read())
print (ag_token_cache)
In [44]:
AGAVE_APP_NAME="funwave-tvd-nectar" + os.environ['AGAVE_USERNAME']
ag = Agave(token=ag_token_cache['access_token'], refresh_token=ag_token_cache['refresh_token'], api_key=ag_token_cache['apikey'], api_secret=ag_token_cache['apisecret'],api_server=ag_token_cache['baseurl'], client_name=AGAVE_APP_NAME, verify=False)
#(api_server=ag_token_cache['baseurl'], api_key=ag_token_cache['apikey'], api_secret=ag_token_cache['apisecret'], verify=False, username=ag_token_cache['username'], password=os.environ.get('AGAVE_PASSWORD'))
The agavepy library's Agave class also provides a restore() method for reconstituting previous OAuth sessions. Previous sessions are read from and written to a cache file, /etc/.agpy, so that OAuth sessions persist across iPython sessions. When you authenticated to JupyterHub, the OAuth login was written to the .agpy file. We can therefore use the restore method to create an OAuth client without needing to pass any credentials:
Note that the restore method can take arguments (such as client_name) so that you can restore/manage multiple OAuth sessions. When first getting started on the hub, there is only one session in the cache file, so no arguments are required.
If we ever want to inspect the OAuth session being used by our client, we have a few methods available to us. First, we can print the token_info dictionary on the token object:
In [46]:
ag.token.token_info
Out[46]:
In [52]:
ag.token.refresh()
Out[52]:
In [53]:
ag.token.token_info
Out[53]:
This shows us both the access and refresh tokens being used. We can also see the end user profile associated with these tokens:
In [54]:
ag.profiles.get()
Out[54]:
Finally, we can inspect the ag object directly for attributes like api_key, api_secret, api_server, etc.
In [55]:
print ag.api_key, ag.api_secret, ag.api_server
We are now ready to interact with Agave services using our agavepy client. We can take a quick look at the available top-level methods of our client:
In [56]:
dir(ag)
Out[56]:
We see there is a top-level method for each of the core science APIs in agave. We will focus on the apps service since it is of broad interest, but much of what we illustrate is generally applicable to all Agave core science APIs.
We can browse a specific collection using the list() method. For example, let's see what apps are available to us:
In [63]:
ag.apps.list()
Out[63]:
What we see in the output above is a python list representing the JSON object returned from Agave's apps service. It is a list of objects, each of which representing a single app. Let's capture the first app object and inspect it. To do that we can use normal Python list notation:
In [64]:
app = ag.apps.list()[0]
In [65]:
print type(app); app
Out[65]:
We see that the app object is of type agavepy.agave.AttrDict. That's a Python dictionary with some customizations to provide convenience features such as using dot notation for keys/attributes. For example, we see that the app object has an 'id' key. We can access it directly using dot notation:
In [66]:
app.id
Out[66]:
Equivalently, we can use normal Python dictionary syntax:
In [67]:
app['id']
Out[67]:
In Agave, the app id is the unique identifier for the application. We'll come back to that in a minute. For now, just know that this example is very typical of responses from agavepy: in general the JSON response object is represented by lists of AttrDicts.
Stepping back for a second, let's explore the apps collection a bit. We can always get a list of operations available for a collection by using the dir(-) method:
In [68]:
dir(ag.apps)
Out[68]:
Also notice that we have tab-completion on these operations. So, if we start typing "ag.apps.l" and then hit tab, Jupyter provides a select box of operations beginning with "l". Try putting the following cell in focus and then hitting the tab key (but don't actually hit enter or try to execute the cell; otherwise you'll get an exception because there's no method called "l"):
In [ ]:
ag.apps.l
If we would like to get details about a specific object for which we know the unique id, in general we use the get method, passing in the id for the object. Here, we will use an app id we found from the ag.apps.list command.
In [70]:
ag.apps.get(appId=app.id)
Out[70]:
Whoa, that's a lot of information. We aren't going to give a comprehensive introduction to Agave applications in this notebook. Instead we refer you to the official Agave app tutorial on the website: http://agaveapi.co/documentation/tutorials/app-management-tutorial/
However, we will point out a couple of important points. Let's capture that response in an object called full_app:
In [71]:
full_app = ag.apps.get(appId=app.id)
Complex sub-objects of the application such as application inputs and parameters come back as top level attributes. and are represented as lists. The individual elements of the list are represented as AttrDicts. We can see this by exploring our full_app's inputs:
In [72]:
print type(full_app.inputs); print type(full_app.inputs[0]); full_app.inputs[0]
Out[72]:
Then, if we want the input id, we can use dot notation or dictionary notation just as before:
In [73]:
full_app.inputs[0].id
Out[73]:
You now have the ability to fully explore individual Agave objects returned from agavepy, but what about searching for objects? The Agave platform provides a powerful search feature across most services, and agavepy supports that as well.
Every retrieval operation in agavepy (for example, apps.list) supports a "search" argument. The syntax for the search argument is identical to that described in the Agave documentation: it uses a dot notation combining search terms, values and (optional) operators. The search object itself should be a python dictionary with strings for the keys and values. Formally, each key:value pair in the dictionary adheres to the following form: $$term.operator:value$$ The operator is optional and defaults to equality ('eq'). For example, the following search filters the list of all apps down to just those with the id attribute equal to our app.id:
In [74]:
ag.apps.list(search={'id': app.id})
Out[74]:
Equivalently, we could explicitly set the equality operator:
In [75]:
ag.apps.list(search={'id.eq': app.id})
Out[75]:
Typically, the list of available search terms is identical to the attributes included in the JSON returned when requesting the full resource description. Operators include 'like', 'lt', 'gt', 'lte', 'gte', etc. See the official Agave documentation for the complete list.
Here we retrieve all apps with a name is "like" opensees:
In [76]:
ag.apps.list(search={'name.like': 'opensees'})
Out[76]:
Two results were returned, both with name "opensees".
You can include multiple search expressions in the form of additional key:value pairs to build a more restrictive query. Here we restrict the result to opensees apps with revision at least 25:
In [77]:
ag.apps.list(search={'name.like': 'opensees', 'revision.gte': 25})
Out[77]:
We hope this gives you enough general information to begin exploring the Agave services using agavepy on your own. In subsequent notebooks, we'll take deeper dives into specific topics such as using agavepy to launch and monitor an Agave job executing OpenSees on Stampede.
In [ ]: